Firefly-1926: optimizing tile generation, transfer, and memory usage #1900
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
robyww
force-pushed
the
FIREFLY-1926-concurrent-grid
branch
from
2d59306 to
9d2c007
Compare
loitly
approved these changes
Jan 13, 2026
Contributor
loitly
left a comment
loitly
left a comment
There was a problem hiding this comment.
The developer walked me through the changes in this PR. It looks like a solid improvement for both the server and client sides. I suggested testing it with one of our most demanding scenario, such as a grid of large images being stretched at the same time. If that works as expected, the PR is good to go.
- Transfer each image tile individually instead of one large data array
- Server not longer copies tiles into a single byte array from tiles, saving server memory
- Client not longer parsees and copies single byte array into tiles, saving client memory
- Client makes a single call per tile and processes the image after it is retrieved, while other tile are being retrieved
- creates better use of worker thread, using CPU/GPU will retrieving a tile
- Client JobRunner act as a batch queue to control how many call are made at a time
- keeps from overwhelming the server with many concurrent calls
- support sending messages from worker
robyww
force-pushed
the
FIREFLY-1926-concurrent-grid
branch
from
9d2c007 to
1ef378f
Compare
Contributor
Author
|
Also tested with Euclid with a full grid of images, about 12.5 GB |
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Firefly-1926: optimizing tile generation, transfer, and memory usage
JobRunner.jsacts as a batch queue to control how many calls are made at a timeTesting